WAND User Trials – Evaluation of a Wireless ATM System

The main objective of the Magic WAND (Wireless ATM Network Demonstrator) project was to develop a wireless ATM demonstrator system and evaluate it in real-life user environment. The output of the project comprised a Wireless ATM Access Network Demonstration system. The developed applications and the WAND hardware was evaluated in different user trials. In this paper we focus on the main results and conclusions of the wireless user trials organised in Tampere, Finland. The wireless trials were conducted with a mobile terminal within laboratory environment. The users were office workers not familiar with the WAND system. We present the performance evaluation of the WAND radio system recorded during the user trials. Additionally, we discuss multipath tolerance measurements performed using a hardware radio channel simulator. Finally we review the subjective user opinions of the Magic WAND system. Introduction Within the ACTS Magic WAND project, a wireless ATM Network Demonstrator has been developed demonstrating that wireless access to ATM, capable of providing real multimedia services to mobile users, is technical feasible. The project was completed by four sets of user trials: medical field trials at Ulm (Germany) and Tampere (Finland) and office field trials at Hildesheim (Germany). For the medical trials at Ulm and Tampere, a standard ATM testbed was used and the emphasis was on evaluating the usefulness and stability of the developed applications. For the mobile computing trials at Hildesheim, students used the WAND terminal to retrieve Web-pages enhanced by various multimedia applets using wireless ATM. Different performance evaluation runs have been performed in order to evaluate the different parts of the software more deeply [6D5]. The fourth set of trials was the wireless user trials in Nokia laboratory in Tampere, Finland. The objective was to evaluate the quality of service of the wireless connection and the functionality of the WAND radio system. Furthermore, the aim of the wireless trials was to assess the differences between the performances of a standard ATM connection and the wireless ATM connection developed in the WAND project. In this paper we present the set-up and the evaluation results of the wireless user trials in Tampere. The analysis is based on measured radio performance data recorded during the user trials and subjective user opinions. Furthermore, the multipath tolerance of the WAND radio system was evaluated outside the user trials by using a hardware radio channel simulator. In the first section we shortly review the objectives of the trials and describe the evaluation process. In the second section we present the radio set-up used in the trials. In the following sections III and IV we discuss the results of the measurements and present the feedback from the test users. Finally, we conclude the results of the wireless trials. Section I: Objectives and the evaluation process The wireless trials were carried out with a full radio subsystem and thus the evaluation was focused on wireless aspects. The main objective was to demonstrate and test the feasibility of a radio based ATM access system in an office environment. The aspects studied were: quality of transmitted data, delays in the connection, stability of the connection and the radio functionality. The users were asked to assess the selected attributes on four types of video connections: • real-time video connection: wireless radio connection versus a standard ATM connection • video playback: wireless radio connection versus a standard ATM connection The trials were conducted with eight office workers who were not familiar with the WAND system, videoconferencing software [KSS98] or application sharing [SKLDP97] system. In addition to the user feedback the actual radio performance was measured and recorded during the trials. The trials were carried out in four sessions. Each session involved two users of which one used the mobile terminal and the other used a wireless local terminal. The users accessed the applications both through the wireless access and the standard ATM connection. A usability expert observed the trials and guided the users in trial assignments if necessary. The users’ subjective opinions were gathered by a questionnaire and an interview after completing the trial tasks. In the questionnaire, the users assessed the quality aspects of the wireless and the wired connections. The questionnaire was designed in co-operation with USINACTS project, a horizontal ACTS project, which supports other ACTS projects in usability issues. The following applications were used to conduct trial tasks: Microsoft NetMeeting, Microsoft Internet Explorer, MS Windows MediaPlayer and MSDOS FTP client. NetMeeting was used for videoconferencing and application sharing. The shared application was Internet Explorer. The MediaPlayer was used for video playback (MPEG format) which were located on the other terminal. A FTP client was used for transferring files from an FTP server to collect statistics for the performance measurements. Section II: Equipment set-up The user trial equipment set-up [LA98] consisted of four PCs (200 MHz). The two application terminals were standard PCs with Windows 95 operating system, Winnov audio / video cards, cameras and Olicom ATM cards. The two radio terminals were HP Vectra PCs with MSDOS 6.22, Olicom ATM cards and WAND radio hardware. A fibre was used to connect the application terminals with the radio terminals in the radio link set up. Application terminals and radio terminals were configured to compose a permanent virtual circuit (PVC) and to use Classical IP over ATM to enable the use of legacy TCP/IP protocol applications. The other application terminal was also connected to the Nokia intranet via a 3Com Ethernet card. There was an internet proxy server installed in this terminal which made access to the Internet possible also from the application terminal which was connected only to its radio terminal. An optimal position was determined for the Access Point (AP) while the Mobile Terminal (MT) was moved between several locations. The entire MT was built into an easily moveable trolley. First location of the MT was in the same room as the AP with a distance of 5 m between AP and MT. Clear line-of-sight (LOS) conditions applied into this set up, although several people moved around in the room blocking the LOS path every now and then. Second location was in a corridor passing the user trial room. Distance between AP and MT in this set up was about 20 m resulting in a non-line-of-sight NLOS situation. Again several people were moving around in both the corridor and in the user trial room. Door leading from the user trial room to the corridor was principally kept shut. However, several people passed through the door while the user trial was going on. User PC #1 User PC #2 AP MT ATM fibers ATM fibers